Exchange Transport Server Architecture (part 1)

The Exchange Transport Server architecture includes several areas whose interdependence and configuration options you should understand.

1. Components of Message Transport

The Transport Pipeline in Exchange Server 2010 consists of several components that work together to route messages. There are four mechanisms to submit a message into the transport pipeline:

• SMTP Protocol SMTP servers or messaging clients can submit mail using the standard SMTP protocol. The message enters the transport pipeline using an SMTP Receive connector.

• Store Driver Mail from Mailbox Servers is submitted via Store Driver.

• Pickup/Replay Folders Properly formatted messages can be dropped into the file system and the server will pick them up and process them.

• Agent/System Generated The system can generate messages such as Non-Delivery Reports (NDRs) or Delivery Reports, and agents can generate messages such as Journal messages.

Figure 1 illustrates the Transport Pipeline and all components of a message transport to route messages through a Hub Transport server. As described later in the chapter, not all of these components are available for Edge Transport servers.

In the figure, the black arrows refer to message delivery or receiving a message; the dotted arrows illustrate how a message is submitted or sent.

1.1. Submission Queue

When the Microsoft Exchange Transport service starts, the categorizer creates one Submission queue on each Edge Transport server and Hub Transport server. The Submission queue stores all messages on disk until the categorizer processes them for further delivery. The categorizer cannot process a message unless a server promotes it to the Submission queue. While the categorizer processes a message, it remains in the Submission queue. After successful categorization, the message is removed from the submission queue.

On the Hub Transport server, message submission can occur through the store driver, through the pickup or replay directory, directly by an agent, or by the Receive connector. On the Edge Transport server, submission is generally only through the Receive connector because messages flow in from the Internet, although pickup or replay directories exist.

1.2. Delivery Queue

Delivery queues hold messages before they are delivered to their target database. Depending on where they should route the messages to, they are called mailbox delivery queue and remote delivery queue.

• Mailbox delivery queues hold messages that are being delivered to a mailbox server located in the same site by using encrypted Exchange RPC. Mailbox delivery queues, one queue for each database, exist on Hub Transport servers only.

• Remote delivery queues hold messages that are being delivered to a remote server by using SMTP. Remote delivery queues can exist on both Hub Transport servers and Edge Transport servers, and more than one remote delivery queue can exist on each server. On an Edge Transport server, these destinations are external SMTP domains or SMTP connectors. On a Hub Transport server, these destinations are SMTP connectors that are outside the Active Directory site in which the Hub Transport server is located or a remote SMTP connector.

1.3. Categorizer

The categorizer retrieves one message at a time from the Submission queue, and always picks the oldest message first. On Hub Transport servers, the categorizer completes the following three steps: recipient resolution, routing resolution, and content conversation. On Edge Transport servers the categorizer puts the message into the delivery queue and routes it to a Hub Transport server.

1.4. Store Driver

Messages sent from mailboxes enter the transport pipeline from the sender's Outbox. After the message is put in the sender's Outbox, the store driver is alerted by the Microsoft Exchange Mail Submission service, the store driver retrieves the message from the sender's Outbox, and then puts it into the Submission queue on a Hub Transport server in the same Active Directory site of the Mailbox server.

The store driver is also responsible for picking up messages that should be delivered locally (delivered in the same Active Directory site) from a delivery queue. The store driver puts the message in the recipient's inbox on the respective Mailbox server.

It is important to understand that store driver is an interface between the Hub server and the Mailbox server. That is why you see it in the beginning and the end of the Transport Pipeline—it is used to for messages submitted by the Mailbox server and to deliver messages to the target mailbox database.

1.5. Microsoft Exchange Mail Submission Service

The Microsoft Exchange Mail Submission service is a notification service running on Mailbox servers. It notifies the store driver on a Hub Transport server in the local Active Directory site when a message is available for retrieval from a sender's Outbox.

1.6. Pickup and Replay Directory

The Pickup and Replay directories serve basically the same purpose: to process properly formatted text files usually generated by applications into messages. The pickup directory is used by applications or users; the Replay directory is used to resubmit exported Exchange messages and to receive messages from foreign gateway servers. Because their purpose is almost identical, this section only focuses on the Pickup directory. Details about the Replay directory can be found at http://technet.microsoft.com/en-us/library/bb124230.aspx.

The Pickup directory is another way for messages to enter the message transport pipeline by being placed into the Pickup directory on a Hub Transport server or an Edge Transport server. The Pickup directory is by default located under <Exchange_Installation_Path>\TransportRoles\Pickup. It can be modified using the following cmdlet:

Set-TransportServer <Servername> -PickupDirectoryPath "C:\Pickup"

The Pickup directory is used to support legacy applications that are not capable of sending messages using MAPI or SMTP protocol but instead by using a simple text file (extension .eml) that is stored in the Pickup directory.

The text file has to include a certain formatting so that Exchange can compose a message out of the text file and sent it. The following requirements must be met:

• The message file must be a text file that complies with the basic SMTP message format. MIME message header fields and content are supported.

• Only one e-mail address can exist in the Sender field. Multiple e-mail addresses aren't allowed.

• At least one e-mail address must exist in the To, Cc, or Bcc fields.

• A blank line must exist between the message header and the message body.

  The following is an example of a valid message that can be processed by the Pickup directory:

To: [email protected]
From: [email protected]
Subject: Pickup Directory Test

Hi!
This is a Pickup Directory Test
Sigi

The pickup process is quite simple: you store a message as a <message>.eml file in the directory, the Pickup directory is checked every five seconds (this cannot be changed), and then .eml files will be processed.

The processing of messages in the pickup directory always follows the same scheme:

2. Message Queues on Transport Servers

Every message that is sent between Transport servers is temporary stored on the Transport server in a place called a message queue. In this location, the message waits for the next step in processing. As explained in Table 1, six different message queues are available on a Transport server. You will find more details on the specific queues later in this chapter.

Exchange 2010 stores all queues in the queue database, which is described in the next section. The queues itself can be managed using either the Queue Viewer or using the Get-Queue cmdlet, as shown in Figure 2.

3. Queue Database

All message queues are stored in a single database called a queue database. The queue database is based on the Extensible Storage Engine (ESE) database technology, which is also used by the Mailbox Server. The queue database is made up of a database file and several log files to record transactions. The checkpoint file tracks which transaction log files have been committed to the database. During a service shutdown of the Microsoft Exchange Transport service, all transaction log files are always committed to the database.

The queue database uses circular logging, which means that the history of committed transactions that are stored in the transaction log files is not maintained. Any transaction log file older than the current checkpoint is immediately and automatically deleted and thus cannot be used to replay queue database recovery.

3.1. Queue Database Files

The queue database consists of several files, which are stored in the following default location: <Exchange_Install_Path>\TransportRoles\data\Queue. Table 2 queue database files and their purpose. provides an overview of the

3.2. Queue Database Configuration Options

Several configuration options exist for the queue database. Unlike Mailbox or public folder databases, settings for the queue database cannot be configured in the Exchange Management Console or with EMS; instead, you need to change the settings in the EdgeTransport.exe.config file located at <Exchange_Server_Install>\Bin. If you change any settings, you need to restart the Microsoft Exchange Transport service for the changes to take effect.

Table 3 provides an overview of options available to configure the queue database.